Method, apparatus and computer-readable recording medium for managing images in image database

ABSTRACT

Managing images in an image database is described, comprising: when a query image is input, performing a match to determine whether an image similar to the query image exists within the database by comparing the images stored in the database with the query image; and if the image similar to the query image is a recognized image, providing at least one image in an image group to which the recognized image belongs and information thereon as a result, and if the image similar to the query image is an unrecognized image, providing at least one image in an image group to which the unrecognized image belongs as a result. When at least one image of the image group including the image similar to the query image is a recognized image, information on the corresponding recognized image is assigned to the images in the image group and provided as a result.

The present patent application is a U.S. National Phase Applicationunder 35 U.S.C. 371 of International Application No. PCT/KR2013/001202filed Feb. 15, 2013, which claims priority from Korean Application No.10-2012-0015543, filed Feb. 15, 2012, the contents of which areincorporated herein in their entirety by reference.

TECHNICAL FIELD

The present disclosure relates to a method, apparatus andcomputer-readable recording medium for managing images in an imagedatabase. More specifically, the present disclosure relates to a method,apparatus and computer-readable recording medium for managing images inan image database, in which at least one image group is formed byassociating the images stored in the image database based on calculatedsimilarities therebetween, comprising, performing a match to determinewhether or not an image similar to an input query image exists withinthe image database by comparing the images previously stored in theimage database with the input query image, if the image similar to thequery image in the match is a recognized image, providing at least oneimage in an image group to which the recognized image belongs andinformation thereon as a result of search, and if the image similar tothe query image in the match is an unrecognized image, providing atleast one image in an image group to which the unrecognized imagebelongs as a result of search, wherein, when at least one image of theimage group including the image determined as being similar to the queryimage is a recognized image, information on the corresponding recognizedimage is assigned to the images included in the image group and isprovided as a result of search.

BACKGROUND ART

Recently, as communication techniques are advanced and use of theInternet is expanded, searches are performed in various ways. Forexample, a computer user may acquire desired information throughInternet searches (further specifically, web services), in addition todirectly consulting a dictionary or asking a person familiar with therelated information, in order to locate the information that thecomputer user desires to know. That is, after connecting to a web serverwhich provides a search service using a web browser, the computer usermay input keywords related to the desired information and receive thesearch service associated with the web server.

The search service is developed into a variety of types. Particularly inKorea, utilization of search for acquisition of information tends toincrease and importance of knowledge search increases. The searchservice is a function used by a large number of users. In addition, itis general that visits of users to a web site is directly related toadvertisement revenues. Thus, a large number of portal sites actuallyprovide search functions.

However, although the number of the sites providing the search functionsincreases, most of the search functions are generally based on a textsearch, such as a keyword search or the like. Thus, they do not satisfydesires of users who want to easily obtain desired information indiverse ways.

Particularly, if the desired information is formed of an image, insteadof text, in a conventional method, a keyword that is supposed to berelated to the corresponding image is first inferred and then, thesearch is performed by inputting the relevant keyword. Although it ispossible to show only the corresponding image results by setting asearch range, the image search itself is merely a kind of text-based(keyword-based) search.

This method has a problem that if a user does not know what acorresponding image relates to, it is difficult to infer a relevantkeyword, and accordingly, the number of inputting keywords for thesearch is increased and the desired information may not be easilysearched for.

Accordingly, in order to solve the problem, an image search system hasbeen developed, in which, when a user desires to obtain information on aspecific image, a search using the image itself, instead of thetext-based search, may be performed.

However, the image search system also has problems as described below.First, in order to provide such an image search system, an imagedatabase containing a sufficient amount of data is required. When asufficient amount of data is not contained in the image database, evenif a query image is input, it may be possible to fail to provide searchresult information for the query image. In addition, all of the imageswhich caused a failed result in matches of the query images and theimages stored in the image database, i.e., unrecognized images, arediscarded, instead of being reused. In this case, until the unrecognizedimages are manually reflected into the image database, despite repeatedsearches, it is impossible to provide a search result for theunrecognized images.

As one of techniques for solving the problems, Korean Patent No.10-1029160 is disclosed by the applicant of the present application. Inthis technique, if the matching between an input query image and imagesstored in the image database end in failure, the query image is storedin an unrecognized image database. In the unrecognized image database,the images having an association relationship are collected into imagegroups based on similarities therebetween. Then, when a specific imageand tag information thereon are input from the outside, at least someimages of a specific image group in the unrecognized image database arecompared with the specific input image to determine whether a similaritytherebetween is equal to or higher than a predetermined threshold value.If it is determined that the similarity therebetween is equal to orhigher than the predetermined threshold value, at least some images inthe unrecognized image database are automatically added, together withthe input tag information, to the recognized image database.

However, this technique is disadvantageous in that its configuration iscomplicated since the unrecognized image database is separately requiredin addition to the image database. In addition, there is a problem inthat the unrecognized images of the image groups stored in theunrecognized image database cannot be used in image recognition andcannot be provided as a search result until a query image (a query imagecontaining tag information) having a similarity with them being equal toor higher than a threshold value is input.

DISCLOSURE OF INVENTION

An object of the present disclosure is to solve at least all theproblems described above.

In addition, another object of the present disclosure is to provide oneor more embodiments directed to allowing unrecognized images to beeffectively used for image search.

Furthermore, still another object of the present disclosure is toprovide one or more embodiments directed to allowing an input queryimage to be compared with images of an unrecognized state in an imagedatabase and to provide relevant information for the query image even ina case where the input query image is determined to have a similaritywith them equal to or higher than a predetermined threshold value.

The representative configuration of the various embodiments of thepresent disclosure for achieving the aforementioned objects is asdescribed below.

According to an aspect of the present disclosure, there is provided amethod of managing images in an image database, which includes the stepsof (a) when a query image is input, performing a match to determinewhether or not an image similar to the query image exists within theimage database by comparing the images previously stored in the imagedatabase with the query image; and (b) if the image determined as beingsimilar to the query image in the match is a recognized image, providingat least one image in an image group to which the recognized imagebelongs and information thereon as a search result, and if the imagedetermined as being similar to the query image in the match is anunrecognized image, providing at least one image in an image group towhich the unrecognized image belongs as a search result, wherein, whenat least one image of the image group including the image determined asbeing similar to the query image is a recognized image, information onthe corresponding recognized image is assigned to the images in theimage group and is provided as a search result.

According to another aspect of the present disclosure, there is provideda method of managing images in an image database, which includes thesteps of (a) when a query image is input, performing a match todetermine whether or not an image similar to the query image existswithin the image database by comparing the query image with the imagespreviously stored in the image database; (b) based on similaritiesbetween the query image and the images stored in the image database, ifthe image determined as being similar to the query image is a recognizedimage, including the query image into an image group to which therecognized image belongs, and if the image determined as being similarto the query image is an unrecognized image, including the query imagein an image group to which the unrecognized image belongs; and (c) whenat least one image of the image group including the image determined asbeing similar to the query image is a recognized image, assigninginformation on the corresponding recognized image to the query imagenewly included in the image group and storing them in the imagedatabase.

According to a further aspect of the present disclosure, there isprovided an apparatus for managing images in an image database, whichincludes the image database for storing at least one image group inwhich images having an association relationship are grouped based onsimilarities between the stored images; and a search unit for, when aquery image is input, performing a match to determine whether or not animage similar to the query image exists within the image database bycomparing the images previously stored in the image database with thequery image, if the image determined as being similar to the query imagein the match is a recognized image, providing at least one image in animage group to which the recognized image belongs and informationthereon as a search result, and if the image determined as being similarto the query image in the match is an unrecognized image, providing atleast one image in an image group to which the unrecognized imagebelongs as a search result, wherein, when at least one image of theimage group including the image determined as being similar to the queryimage is a recognized image, information on the corresponding recognizedimage is assigned to the images in the image group and is provided as asearch result.

According to a still further aspect of the present disclosure, there isprovided an apparatus for managing images in an image database, whichincludes a search unit for, when a query image is input, performing amatch to determine whether or not an image similar to the query imageexists within the image database by comparing the query image with theimages previously stored in the image database; and an image groupforming unit for, based on similarities between the query image and theimages stored in the image database, if the image determined as beingsimilar to the query image is a recognized image, including the queryimage into an image group to which the recognized image belongs and ifthe image determined as being similar to the query image is anunrecognized image, including the query image in an image group to whichthe unrecognized image belongs, when at least one image of the imagegroup including the image determined as being similar to the query imageis a recognized image, assigning information on the correspondingrecognized image to the query image newly included in the image groupand storing them in the image database.

In addition, there is further provided other methods, apparatuses, and acomputer-readable recording medium for recording a computer program forexecuting the aforementioned methods for implementing the presentdisclosure.

According to various embodiments of the present disclosure, theconfiguration of the apparatus can be simplified since it is unnecessaryto separately construct an image database for storing recognized imagesand an unrecognized image database for storing unrecognized images.

In addition, according to some embodiments of the present disclosure,even if an input query image is compared with images of an unrecognizedstate in the image database, if it is determined that a similaritytherebetween is equal to or higher than a predetermined threshold value,relevant information for the query image can be provided so that a usermay be provided with a desired search result further abundantly.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view schematically showing the configuration of an entiresystem for managing an image database according to some embodiments.

FIG. 2 is a view showing the internal configuration of an imageprocessing apparatus 200 according to some embodiments.

FIGS. 3a to 3c are views showing that image groups are formed bycombining a query image with images previously stored in an imagedatabase, which have high similarities with the query image, accordingto some embodiments.

FIG. 4 is a view showing an image group forming unit that collectsimages similar to each other from images stored in an image database toform an image group, according to some embodiments.

FIGS. 5a to 5d are views showing the configuration of normalizing afeature region according to some embodiments.

FIGS. 6a to 6b are views showing distribution of feature regionscontained in an image collected by a web crawler and an image belongingto an image group according to some embodiments.

BEST MODE FOR CARRYING OUT THE INVENTION

In the following detailed description of various embodiments of thepresent disclosure, references are made to the accompanying drawingsthat show, by way of illustration, specific embodiments in which thepresent disclosure may be implemented. These embodiments are describedin sufficient detail to enable those skilled in the art to implement thepresent disclosure. It should be understood that various embodiments ofthe present disclosure, although different, are not necessarily mutuallyexclusive. For example, specific feature, structure, and characteristicdescribed herein, in connection with one embodiment, may be implementedwithin other embodiments without departing from the spirit and scope ofthe present disclosure. In addition, it should be understood that thelocation or arrangement of individual elements within each disclosedembodiment may be modified without departing from the spirit and scopeof the present disclosure. The following detailed description is,therefore, not to be taken in a limiting sense, and the scope of thepresent disclosure is defined only by the appended claims, appropriatelyinterpreted, along with the full range equivalent to what the claimsclaim. In the drawings, like reference numbers refer to the same orsimilar function through many ways.

Hereinafter, various embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings in orderto easily implement the present disclosure by those skilled in the art.

FIG. 1 is a view schematically showing the configuration of an entiresystem for managing an image database according to embodiments.

As shown in FIG. 1, the entire system according to some embodiments mayinclude a communication network 100, an image processing apparatus 200and user terminal devices 300.

First, the communication network 100 can be configured regardless of itscommunication scheme, such as wired or wireless communication and mayinclude a variety of communication networks such as a local area network(LAN), a metropolitan area network (MAN), a wide area network (WAN) andthe like. The communication network 100 referred to in the presentdisclosure may be a well known world wide web (WWW).

According to some embodiments, in a state where at least one image groupis formed in an image database (not shown) by associating images storedin the image database based on calculated similarities therebetween, theimage processing apparatus 200 may perform a match to determine whetheror not an image similar to a query image exists within the imagedatabase by comparing the images previously stored in the image databasewith the input query image. If the image determined as being similar tothe query image in the match is a recognized image, at least one imagein an image group to which the recognized image belongs and informationthereon are provided as a search result. If the image determined asbeing similar to the query image in the match is an unrecognized image,at least one image in an image group to which the unrecognized imagebelongs is provided as a search result. When at least one image of theimage group including the image determined as being similar to the queryimage is a recognized image, information on the corresponding recognizedimage is assigned to the images belonging to the image group and isprovided as a search result.

In addition, according to some embodiments, in addition to when a queryis input as described above, the image processing apparatus 200 maycalculate periodically or aperiodically similarities between the imagesstored in the image database and may make a group with the images havingsimilarities therebetween equal to or higher than a predeterminedthreshold value. If a recognized image exists within the image group,the image processing apparatus 200 may update the image database byassigning information on the recognized image to images within the sameimage group.

In addition, according to some embodiments, when a query image is input,the image processing apparatus 200 may perform a match to determinewhether or not an image similar to the query image exists within theimage database by comparing the query image with the images previouslystored in the image database, and then, based on similarities betweenthe query image and the images stored in the image database, if theimage similar to the query image is a recognized image, includes thequery image into an image group to which the recognized image belongs,and, if the image similar to the query image is an unrecognized image,includes the query image into an image group to which the unrecognizedimage belongs. When at least one image of the image group including theimage determined as being similar to the query image is a recognizedimage, information on the corresponding recognized image may be assignedto the query image newly included in the image group and may be storedin the image database.

Meanwhile, the user terminal device 300 according to some embodiments isa digital device having a function of allowing a user to connect to andcommunicate with the image processing apparatus 200. Any digital deviceprovided with a memory means and equipped with a microprocessor and thushaving operation capability, such as a personal computer (e.g., adesktop computer, a notebook computer or the like), a workstation, aPDA, a web pad, a cellular phone or the like, can be adopted as the userterminal device 300.

Hereinafter, the internal configuration of the image processingapparatus 200 which performs an important function in order to implementvarious embodiments of the present disclosure and the functions ofrespective constitutional components will be described.

FIG. 2 is a view showing the internal configuration of an imageprocessing apparatus 200 according to some embodiments.

Referring to FIG. 2, the image processing apparatus 200 according tosome embodiments may includes a communication unit 210, a control unit220, a search unit 230, an image group forming unit 240, an additionaldata acquisition and comparison unit 250 and an image database 260.

According to some embodiments, at least some of the communication unit210, the control unit 220, the search unit 230, the image group formingunit 240, the additional data acquisition and comparison unit 250, andthe image database 260 may be program modules communicating with theuser terminal device 300. Such program modules may be included in theimage processing apparatus 200 in a form of an operating system, anapplication program module and other program modules, and physically,they may be stored in a variety of well known memory devices. Inaddition, these program modules may be stored in a remote memory devicecapable of communicating with the image processing apparatus 200.Meanwhile, although such program modules include routines, subroutines,programs, objects, components, data structures and the like, whichperform specific tasks to be described below or execute specificabstract data types, they are not limited thereto.

First, according to some embodiments, the image database 260 may includeat least one image group of images associated and grouped based onsimilarities therebetween.

According to some embodiments, if a query image is input, the searchunit 230 may perform a function of determining whether or not an imagesimilar to the query image (i.e., an image having a similarity with thequery image being equal to or higher than a predetermined thresholdvalue) exists in the image database 260 by matching the query image tothe images stored in the image database 260.

Specifically, according to some embodiments, the search unit 230 mayperform a function of determining whether or not an image similar to thequery image exists within the image database 260 by matching anormalized feature region of an image stored in the image database 260to a normalized feature region of the query image.

In addition, if an image matching to the query image is a recognizedimage stored in the image database 260 (for example, an image to whichinformation on the corresponding image is assigned in a form of a tag),the search unit 230 provides at least one image in an image group towhich the corresponding recognized image belongs and information thereonas a result of search, and if the image matching to the query image isan unrecognized image stored in the image database 260 (for example, animage to which information on the corresponding image is not assigned ina form of a tag), the search unit 230 provides at least one image in animage group to which the corresponding unrecognized image belongs andinformation thereon as a result of search. If at least one image of theimage group including the image determined as being similar to the queryimage is a recognized image, the search unit 230 may assign informationon the corresponding recognized image to the images belonging to theimage group and provide them as a result of search.

Here, a feature point and a feature region may be previously extractedfrom an image for the matching between the images. Here, the featurepoint means a point that contains a feature element of an objectcontained in a corresponding image, and the feature region means aregion around the feature point containing the feature of the object,which may be set to be robust to the changes in illumination andviewpoint of the image.

As described above, a certain feature extraction technique is requiredin order to extract a feature point and a feature region from an image.According to some embodiments, as a feature extraction technique,reference may be made to C. Harris et al., “A combined corner and edgedetector,” Alvey Vision Conference, 1988, or the like (which should beconsidered as being incorporated herein by reference in its entirety).In this paper, described is a method of estimating a feature region ofan oval shape using a second moment matrix, which expresses a slopedistribution around a feature point. Of course, an object recognitiontechnique applicable to the present disclosure is not limited only tothe method described in the aforementioned paper, but the presentdisclosure can be implemented by applying various modified examples.

In addition, based on similarities between the images stored in theimage database 260, the image group forming unit 240 according to ansome embodiments may perform a function of forming an image group bycombining images highly associated (for example, having similaritiesbetween the images being equal to or higher than a predetermined value).Further specifically, the image group forming unit 240 may perform thefunction of forming an image group by comparing feature points orfeature regions of the images stored in the image database 260 and thencombining images that are determined to be similar to each other.

The image group forming unit 240 may perform the function of forming animage group, periodically or aperiodically, in addition to when a queryimage is input. It is possible that if at least one image of each imagegroup is a recognized image, the image group forming unit 240 may assigninformation on the recognized image, such as a tag, to images of thesame image group (so-called a tag propagation). In addition, it is alsopossible to form and keep a cluster where the tag propagation has notbeen performed, i.e., where recognized images and unrecognized imagesare mixed in the same image group, and to perform the tag propagation ifnecessary in the future.

Additionally, the image group forming unit 240 may cause the imagesstored in the image database 260 to be included in one or more imagegroups. For example, when an image stored in the image database 260contains two objects, for example, a rose and a pine tree, the image canbe commonly included in an image group including an image which containsa rose and an image group including an image which contains a pine tree.

Meanwhile, when it is determined by the search unit 230 that an imagematching to the query image exists within the image database 260, theimage group forming unit 240 may perform a function of creating anupdated image group by combining the query image into an image group inthe image database 260 to which the matching image belongs. At thispoint, it is apparent that the tag propagation in the updated imagegroup may be performed at an appropriate time point.

In addition, even when the additional data acquisition and comparisonunit 250, which will be described below, acquires a certain image andcorresponding information and determines that an image matching to thecertain image exists within the image database 260, the image groupforming unit 240 may perform a function of creating an updated imagegroup by combining the certain image into an image group in the imagedatabase 260, to which the corresponding matching image belongs. Even atthis point, it is apparent that the tag propagation in the updated imagegroup may be performed at an appropriate time point.

FIGS. 3a and 3b show creating a cluster in a case where a certain imagegroup previously stored in the image database 260 contains three imageshaving similarities therebetween equal to or higher than a thresholdvalue of 0.7.

In FIGS. 3a and 3b , circles with K1, K2 and K3 respectively writteninside thereof may represent recognized images, circles with U1, U2 andU3 respectively written inside thereof represent unrecognized images,and a circle with a Q written inside thereof represents a query image.In addition, each of circles with a T1 written inside thereof mayrepresent information on the image, i.e., a tag. Lines between theimages respectively may represent connection relationships, and thenumerals written on the lines represent similarities. OIM1, OIM2 andOIM3 may represent previous image groups, and NIM1, NIM2 and NIM3represent new image groups. In addition, the dotted circles with T1written inside thereof may represent a tag assigned through the tagpropagation.

Referring to FIG. 3a , when a query image Q is input, the search unit230 acquires a similarity of 0.9 by calculating a similarity between thequery image Q and the recognized image K2 stored in the image database260. Then, the image group forming unit 240 may create an associationrelationship between the query image Q and the recognized image K2stored in the image database 260, include the query image in a new imagegroup NIM1, and store the new image group in the image database 260. Inaddition, the image group forming unit 240 may assign a tag T1 relatedto the images K1, K2 and K3 of the image database 260 to the query imageQ. Then, the search unit 230 may provide the recognized images K1, K2and K3 of the new image group NIM1 and information T1 thereon as aresult of search. FIG. 3a shows that the similarity between therecognized image K2 of the previous recognized image group OIM1 and theunrecognized query image Q is higher than 0.7 to thereby create anassociation relationship and the tag T1 is assigned to the unrecognizedquery image Q to covert the unrecognized query image into a recognizedquery image so that a new recognized image group NIM1 is created.

Referring to FIG. 3b , when a query image Q is input, the search unit230 acquires a similarity of 0.7 by calculating a similarity between thequery image Q and the unrecognized image U2 stored in the image database260. Then, the image group forming unit 240 may create an associationrelationship between the query image Q and the unrecognized image U2stored in the image database 260 to create a new image group NIM2, andstore the new image group in the image database 260. Then, the searchunit 230 may provide the images U1, U2 and U3 of the new image groupNIM2 as a result of search. FIG. 3b shows that a similarity between theunrecognized images U1, U2 and U3 of the previous unrecognized imagegroup OIM1 and the query image Q is higher than 0.7 to thereby create anassociation relationship so that a new unrecognized image group NIM2 iscreated.

Referring to FIG. 3c , when a query image Q is input, the search unit230 acquires a similarity of 0.7 by calculating a similarity between thequery image Q and the unrecognized image U2 stored in the image database260. Then, the image group forming unit 240 may create an associationrelationship between the query image Q and the unrecognized image U2stored in the image database 260 to create a new image group NIM3, andstore the new image group in the image database 260. Then, the searchunit 230 may not only provide the images K1, U2 and U3 of the new imagegroup NIM3 as a result of search but also assign a tag T1 related to therecognized image K1 included in the new image group NIM3 to the queryimage Q and the unrecognized images U2 and U3 to additionally providethem. FIG. 3c shows that a similarity between the recognized image K2and unrecognized images U2 and U3 of the previous unrecognized imagegroup OIM3 and the query image Q is equal to or higher than 0.7 tothereby create an association relationship so that a new recognizedimage group NIM3 is created.

In addition, FIG. 4 shows an embodiment of forming an image group bycombining images having a high similarity with each other from theimages stored in the image database 260 by the image group forming unit240 according to some embodiments. Referring to FIG. 4, the image groupforming unit 240 according to some embodiments collects imagesdetermined as being similar to each other from the image database 260 asshown in FIG. 4 using an image matching technique. Next, the image groupforming unit 240 according to some embodiments may store the imagesarranged and structured by time and diverse viewpoints in order to formthe images collected as shown in FIG. 4 into a structured image group.Although FIG. 4 shows an embodiment of arranging and structuring imagesby time and diverse viewpoints by the image group forming unit 240, thepresent disclosure is not limited thereto. The image group forming unit240 may form an image group in a variety of methods.

In addition, the additional data acquisition and comparison unit 250 mayacquire a specific image and tag information thereon from outside (forexample, a web crawler or a user terminal device) and determine whetheror not a similarity between at least some of images in an image group ofthe image database 260 and the specific input image is equal to orhigher than a predetermined threshold value by comparing the at leastsome of the images and the specific input image. If the additional dataacquisition and comparison unit 250 determines that a certain image inthe image group has a similarity equal to or higher than thepredetermined threshold value, the image group forming unit 240 mayupdate a certain image group, to which the certain image belongs, sothat the specific image input from the outside and the tag informationthereon may be included in the certain image group included in the imagedatabase 260.

For reference, the additional data acquisition and comparison unit 250may acquire images collected by the web crawler or the like or an imageinput from the user terminal device together with tag information,excluding the query image (which is assumed not to include taginformation, but is not necessarily limited thereto) as additional data,and compare it with the images in the image database 260.

FIGS. 5a to 5d are views showing the configuration of normalizing afeature region according to some embodiments. An image of FIG. 5a is apart of an image collected by a web crawler or the like, which shows afeature region A 510 extracted from the image collected by the webcrawler or the like, and an image of FIG. 5b shows a feature region B520 extracted from an image belonging to an image group. Observing theimages of FIGS. 5a and 5b , it can be confirmed that although the imagesare actually images of the same object, the same feature part of thesame object are displayed differently in size and shape in therespective images due to the difference in viewpoint and illumination.Accordingly, it can be confirmed that feature regions 510 and 520 havingdifferent sizes and shapes are extracted from the same feature parts ofan image collected by the web crawler or the like and an image belongingto an image group. The additional data acquisition and comparison unit250 may normalize a pair of feature regions having different sizes andshapes into a pair of feature regions having the same size and shapeusing a certain normalization technique. Accordingly, the pair offeature regions 510 and 520, such as the images of FIGS. 5a and 5b canbe normalized into a pair of feature regions 530 and 540, such as imagesof FIGS. 5c and 5 d.

Meanwhile, as described above, a certain feature region normalizationtechnique is required for normalizing a feature region extracted from animage. According to some embodiments, as a feature region normalizationtechnique, reference may be made to K. Mikolajczyk et al., “A Comparisonof Affine Region Detectors,” International Journal of Computer Vision,September, 2005, or the like (which should be considered as beingincorporated herein by reference in its entirety). In this paper,described is a method of normalizing a feature region formed in anellipse having various sizes and shapes into a circle having a specificsize and shape using M_(L) ^(1/2) and M_(R) ^(1/2) of a second momentmatrix which estimates viewpoints and lighting conditions of acorresponding image, and a method of rotating a normalized featureregion using a rotation matrix R in order to determine whether or not apair of normalized feature regions indicate the same object. Of course,the normalization technique applicable to the present disclosure is notlimited to the method described in the aforementioned paper, but thepresent disclosure can be implemented by applying various modifiedexamples.

In addition, the additional data acquisition and comparison unit 250 maycompare at least one normalized feature region of an image belonging toan image group with at least one normalized feature region of thecollected image to search for at least a pair of feature regionsdetermined as indicating the same object from respective images.Furthermore, when at least two pairs of feature regions are searched,the additional data acquisition and comparison unit 250 may compare arelative positional relationship between at least two feature regions onthe image belonging to an image group corresponding to the at least twopairs of feature regions and a relative positional relationship betweenat least two feature regions on the collected image corresponding to theat least two pairs of feature regions using a topology and determinewhether or not the image belonging to the image group matches thecollected image with reference to a result of the comparison.

FIGS. 6a to 6b are views showing a distribution of feature regionscontained in an image collected by a web crawler or the like and animage, belonging to an image group according to some embodiments.Referring to FIGS. 6a to 6b , the additional data acquisition andcomparison unit 250 may determine whether or not two images are similarto each other by comparing a relative positional relationship of aplurality of feature regions on an image collected by a web crawler orthe like (i.e., an image of FIG. 6a ) with a relative positionalrelationship of a plurality of feature regions on an image belonging toan image group (i.e., an image of FIG. 6b ).

Meanwhile, as described above, a certain topology technique is requiredfor determining whether or not two different images are similar to eachother using a relative positional relationship of feature regions.According to some embodiments, as such a topology technique, referencemay be made to Derdar Salah et al., “Image matching using algebraictopology,” Proceedings of SPIE, Vol. 6066, January, 2006, or the like(which should be considered as being incorporated herein by reference inits entirety). In this paper, described is a method of performing imagematching by measuring a similarity between boundary elements of afeature contained in an image using an algebraic topology. Of course,the topology technique applicable to the present disclosure is notlimited to the method described in the aforementioned paper, but thepresent disclosure can be implemented by applying various modifiedexamples.

As described above, according to the image matching method of thepresent disclosure, there is an effect in that accuracy of imagematching between an image collected by a web crawler or the like and animage belonging to an image group can be improved.

Meanwhile, when a new image and information thereon are added to theimage database 260, the additional data acquisition and comparison unit250 may allow both n representative images representing the added imageand representative tags assigned to the representative images to beadded. In addition, when the representative images are added, theadditional data acquisition and comparison unit 250 may also add msub-images in different viewpoints and time from those of therepresentative images, together with the representative images. Further,sub-tags for the sub-images added in this case may also be selectivelyincluded and added. Here, the representative images and the sub-imagesrefer to images in a relation of sharing one or more feature points orfeature regions.

Meanwhile, the image database 260 is a concept including also a databaseof a broad sense including data records or the like based on a computerfile system, as well as a database of a narrow sense. It should beunderstood that the database mentioned in the present disclosure mayinclude even a collection of operational processing logs if certain datacan be extracted by searching the logs. Although it is shown in FIG. 2that the image database 260 is included in the image processingapparatus 200, the image database 260 may be configured to be separatefrom the image processing apparatus 200 according to the needs of thoseskilled in the art implementing the present disclosure.

Meanwhile, the communication unit 210 performs a function of allowingthe image processing apparatus 200 to communicate with an externaldevice such as the user terminal device 300.

In addition, the control unit 220 performs a function of controllingdata flow between the communication unit 210, the search unit 230, theimage group forming unit 240, the additional data acquisition andcomparison unit 250, and the image database 260. That is, the controlunit 220 controls the flow of data from outside or between theconstitutional components of the image processing apparatus 200, so thatthe communication unit 210, the search unit 230, the image group formingunit 240, the additional data acquisition and comparison unit 250, andthe image database 260 may perform their respective unique functions.

Meanwhile, the entire image processing apparatus 200 or at least somecomponents thereof, such as communication unit 210, the search unit 230,the image group forming unit 240, the additional data acquisition andcomparison unit 250, and the image database 260 can be implemented by acloud computing server which virtualizes a plurality of serverapparatuses sharing computing resources or server resources.

The above-described embodiments of the present disclosure can beimplemented as computer readable codes in a computer readable medium.The computer readable recording medium may include a programinstruction, a local data file, a local data structure, or a combinationthereof. The computer readable recording medium may be specific tovarious embodiments of the present disclosure or commonly known to thoseof ordinary skill in computer software. The computer readable recordingmedium includes all types of recordable media in which computer readabledata are stored. Examples of the computer readable recording mediuminclude a magnetic medium, such as a hard disk, a floppy disk and amagnetic tape, an optical medium, such as a CD-ROM and a DVD, amagneto-optical medium, such as a floptical disk, and a hardware memory,such as a ROM, a RAM and a flash memory, specifically configured tostore and execute program instructions. Examples of the programinstruction include machine code, which is generated by a compiler, anda high level language, which is executed by a computer using aninterpreter and so on. The above-described hardware apparatus may beconfigured to operate as one or more software modules for performing theoperation of the present disclosure, and the reverse case is similar.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the disclosures. Indeed, the novel methods and apparatusesdescribed herein may be embodied in a variety of other forms;furthermore, various changes, modifications, corrections, andsubstitutions with regard to the embodiments described herein may bemade without departing from the spirit of the disclosures.

Therefore, the accompanying claims and their equivalents including theforegoing modifications are intended to cover the scope and spirit ofthe disclosures, and are not limited by the present disclosures.

The invention claimed is:
 1. A method of managing images in an imagedatabase, comprising: (a) performing a match when an image is input as aquery image to determine whether or not an image similar to the queryimage exists within the image database by comparing the imagespreviously stored in the image database with the query image; and (b)providing at least one image in an image group to which an recognizedimage belongs and information thereon as a search result if the imagedetermined as being similar to the query image in the match is therecognized image, and providing at least one image in an image group towhich an unrecognized image belongs as a search result if the imagedetermined as being similar to the query image in the match is theunrecognized image, wherein, when at least one image of the image groupincluding the image determined as being similar to the query image is arecognized image, information on the corresponding recognized image isassigned to the images in the image group and is provided as a searchresult, wherein the recognized image is an image to which informationthereon is assigned and the unrecognized image is an image to whichinformation thereon is not assigned, (c) updating the image database,wherein, when at least two pairs of feature regions are searched for,updating the image database includes: comparing a relative positionalrelationship between at least two feature regions on an image belongingto the certain image group corresponding to the at least two pairs offeature regions with a relative positional relationship between at leasttwo feature regions on the specific input image corresponding to the atleast two pairs of feature regions using a topology, and determiningwhether or not the image belonging to the certain image group matchesthe specific input image, wherein, when a specific image and taginformation thereon are input from outside, updating the image databaseincludes: adding the specific image and the tag information thereoninput from the outside to a certain image group if it is determined thata certain image in the certain image group has a similarity with thespecific image being equal to or higher than a predetermined thresholdvalue.
 2. The method according to claim 1, further comprising, when thespecific image and tag information thereon are input from outside,comparing at least some images in the certain image group with thespecific image to determine whether or not similarities between the atleast some images in the certain image group and the specific image areequal to or higher than a predetermined threshold value.
 3. The methodaccording to claim 2, wherein the specific image and the tag informationthereon are received from a crawler or a user terminal device.
 4. Themethod according to claim 2, wherein updating the image database furtherincludes: determining similarities between the specific input image andthe images of the certain image group by comparing at least onenormalized feature region of the images of the certain image group withat least one normalized feature region of the specific input image andsearching for at least a pair of feature regions determined asindicating a same object from the respective images.
 5. The methodaccording to claim 2, wherein at least one or more steps of the steps(a), (b) and (c) are executed by a cloud computing server, the cloudcomputing server virtualizing a plurality of server apparatuses sharingcomputing resources or server resources.
 6. The method according toclaim 1, further comprising, based on similarities between the imagesstored in the image database, grouping images having similarities witheach other being equal to or higher than the predetermined thresholdvalue to thereby form an image group.
 7. The method according to claim6, wherein grouping images further includes: determining thesimilarities by comparing feature points or feature regions of theimages stored in the image database and forming the image groupaccording to the similarities being determined.
 8. The method accordingto claim 1, wherein the step (a) further comprises, when the query imageis input, matching normalized feature regions of the images previouslystored in the image database to a normalized feature region of the queryimage to determine whether or not an image similar to the query imageexists.
 9. The method according to claim 1, wherein updating the imagedatabase further includes: adding representative images representing theat least some images in the certain image group and representative tagscorresponding to the representative images to the image database. 10.The method according to claim 9, wherein m sub-images in differentviewpoints and time from those of the representative images are addedtogether with the representative images.
 11. The method according toclaim 10, wherein when the sub-images are added, sub-tags for thesub-images are further added.
 12. The method according to claim 10,wherein the representative images and the sub-images share one or morefeature points or feature regions.
 13. A method of managing images in animage database, comprising: (a) performing a match when an image isinput as a query image to determine whether or not an image similar tothe query image exists within the image database by comparing the queryimage with the images previously stored in the image database; (b) basedon similarities between the query image and the images stored in theimage database, including the query image into an image group to which arecognized image belongs if the image determined as being similar to thequery image is the recognized image, and including the query image in animage group to which an unrecognized image belongs if the imagedetermined as being similar to the query image is the unrecognizedimage; (c) assigning information on the corresponding recognized imageto the query image newly included in the image group and storing thequery image and the information assigned to the query image in the imagedatabase when at least one image of the image group including the imagedetermined as being similar to the query image is the recognized image,wherein the recognized image is an image to which information thereon isassigned and the unrecognized image is an image to which informationthereon is not assigned; and (d) updating the image database, wherein,when at least two pairs of feature regions are searched for, updatingthe image database includes: comparing a relative positionalrelationship between at least two feature regions on an image belongingto the certain image group corresponding to the at least two pairs offeature regions with a relative positional relationship between at leasttwo feature regions on the specific input image corresponding to the atleast two pairs of feature regions using a topology, and determiningwhether or not the image belonging to the certain image group matchesthe specific input image, and wherein, when a specific image and taginformation thereon are input from outside, updating the image databaseincludes: adding the specific image and the tag information thereoninput from the outside to a certain image group if it is determined thata certain image in the certain image group has a similarity with thespecific image being equal to or higher than a predetermined thresholdvalue.
 14. An apparatus for managing images in an image database,comprising: the image database for storing at least one image group inwhich images having an association relationship are grouped based onsimilarities between the stored images; and a memory to storeinstructions thereon, wherein when the processor executes theinstructions, the processor is caused to: a processor, wherein when theprocessor executes instructions stored on a memory, the processorperforms a method including: performing a match when an image is inputas a query image to determine whether or not an image similar to thequery image exists within the image database by comparing the imagespreviously stored in the image database with the query image, providingat least one image in an image group to which a recognized image belongsand information thereon as a search result if the image determined asbeing similar to the query image in the match is the recognized image,and providing at least one image in an image group to which anunrecognized image belongs as a search result if the image determined asbeing similar to the query image in the match is the unrecognized image,wherein, when at least one image of the image group including the imagedetermined as being similar to the query image is a recognized image,information on the corresponding recognized image is assigned to theimages in the image group and is provided as a search result, whereinthe recognized image is an image to which information thereon isassigned and the unrecognized image is an image to which informationthereon is not assigned, and when at least two pairs of feature regionsare searched for, comparing a relative positional relationship betweenat least two feature regions on an image belonging to the certain imagegroup corresponding to the at least two pairs of feature regions with arelative positional relationship between at least two feature regions onthe specific input image corresponding to the at least two pairs offeature regions using a topology and determining whether or not theimage belonging to the certain image group matches the specific inputimage.
 15. The apparatus according to claim 14, wherein the processorperforms a method further comprising: when the specific image and taginformation thereon are input from outside, comparing at least someimages in a certain image group with the specific image to determinewhether or not similarities between the at least some images in thecertain image group and the specific image are equal to or higher thanthe predetermined threshold value.
 16. The apparatus according to claim15, wherein the processor performs a method further comprising:receiving the specific image and the tag information thereon from acrawler or a user terminal device.
 17. The apparatus according to claim16, wherein the processor determines similarities between the specificinput image and the images of the certain image group by comparing atleast one normalized feature region of the images of the certain imagegroup with at least one normalized feature region of the specific inputimage and searching for at least a pair of feature regions determined asindicating the same object from the respective images.
 18. The apparatusaccording to claim 14, wherein the processor performs a method furthercomprising: based on similarities between the images stored in the imagedatabase, grouping images having similarities with each other beingequal to or higher than the predetermined threshold value to therebyform an image group.
 19. The apparatus according to claim 18, whereinthe processor determines the similarities by comparing feature points orfeature regions of the images stored in the image database, and formsthe image group based on the similarities.
 20. The apparatus accordingto claim 14, wherein, when the query image is input, the processormatches normalized feature regions of the images previously stored inthe image database to a normalized feature region of the query image todetermine whether or not an image similar to the query image exists. 21.The apparatus according to claim 14, wherein the processor adds nrepresentative images representing the at least some images in thecertain image group and representative tags corresponding to therepresentative images to the image database.
 22. The apparatus accordingto claim 21, wherein m sub-images in different viewpoints or time fromthose of the representative images are added together with therepresentative images.
 23. The apparatus according to claim 22, whereinthe representative images and the sub-images share one or more featurepoints or feature regions.